Following Gaze Across Views
نویسندگان
چکیده
Following the gaze of people inside videos is an important signal for understanding people and their actions. In this paper, we present an approach for following gaze across views by predicting where a particular person is looking throughout a scene. We collect VideoGaze, a new dataset which we use as a benchmark to both train and evaluate models. Given one view with a person in it and a second view of the scene, our model estimates a density for gaze location in the second view. A key aspect of our approach is an end-to-end model that solves the following sub-problems: saliency, gaze pose, and geometric relationships between views. Although our model is supervised only with gaze, we show that the model learns to solve these subproblems automatically without supervision. Experiments suggest that our approach follows gaze better than standard baselines and produces plausible results for everyday situations.
منابع مشابه
A Novel Approach for Driver Eye Gaze Estimation using Epipolar Geometry
The objective of this research is to estimate the gaze point of a driver using epipolar geometry. The system requires two video inputs from internal and external views of a vehicle. We use SIFT image descriptor with L-2 norm distance to find putative correspondences, and then we use RANSAC algorithm to estimate robust fundamental matrix between two views. Once the underlaying scene geometry is ...
متن کاملRhesus monkeys show human-like changes in gaze following across the lifespan.
Gaze following, or co-orienting with others, is a foundational skill for human social behaviour. The emergence of this capacity scaffolds critical human-specific abilities such as theory of mind and language. Non-human primates also follow others' gaze, but less is known about how the cognitive mechanisms supporting this behaviour develop over the lifespan. Here we experimentally tested gaze fo...
متن کاملLooking past the model species: diversity in gaze-following skills across primates.
Primates must navigate complex social landscapes in their daily lives: gathering information from and about others, competing with others for food and mates, and cooperating to obtain rewards as well. Gaze-following often provides important clues as to what others see, know, or will do; using information about social attention is thus crucial for primates to be competent social actors. However,...
متن کاملAltered gaze following during live interaction in infants at risk for autism: an eye tracking study
BACKGROUND The ability to follow gaze is an important prerequisite for joint attention, which is often compromised in children with autism spectrum disorder (ASD). The direction of both the head and eyes provides cues to other people's attention direction, but previous studies have not separated these factors and their relation to ASD susceptibility. Development of gaze following typically occu...
متن کاملNew perspectives in gaze sensitivity research.
Attending to where others are looking is thought to be of great adaptive benefit for animals when avoiding predators and interacting with group members. Many animals have been reported to respond to the gaze of others, by co-orienting their gaze with group members (gaze following) and/or responding fearfully to the gaze of predators or competitors (i.e., gaze aversion). Much of the literature h...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1612.03094 شماره
صفحات -
تاریخ انتشار 2016